Skip to content

feat(db): add MDBX put-append for fast ordered puts #18603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 24, 2025
Merged

feat(db): add MDBX put-append for fast ordered puts #18603

merged 1 commit into from
Sep 24, 2025

Conversation

hai-rise
Copy link
Contributor

@hai-rise hai-rise commented Sep 21, 2025 •
edited
Loading

We only want 2 things in life:

  1. Flush blocks to disk as fast as possible.
  2. Commit write transactions as fast as possible.

This led us to DatabaseProvider::insert_block, which is still using put-upsert instead of put-append for tables with naturally increasing keys (like BlockNumber) in the happy/hot paths. For example:

reth/crates/storage/provider/src/providers/database/provider.rs

Lines 2747 to 2755 in 4ddf3dd

self.tx.put::<tables::CanonicalHeaders>(block_number, block.hash())?;
durations_recorder.record_relative(metrics::Action::InsertCanonicalHeaders);
// Put header with canonical hashes.
self.tx.put::<tables::Headers<HeaderTy<N>>>(block_number, block.header().clone())?;
durations_recorder.record_relative(metrics::Action::InsertHeaders);
self.tx.put::<tables::HeaderTerminalDifficulties>(block_number, ttd.into())?;
durations_recorder.record_relative(metrics::Action::InsertHeaderTerminalDifficulties);

This PR first adds DbTxMut::append with the optimised implementation for MDBX, and a benchmark to show how put-append is much faster than put-upsert in this scenario:

$ cargo bench -p reth-db --features test-utils --bench put
Benchmarking Put/put: Warming up for 3.0000 s
Put/put                 time:   [138.67 ms 138.76 ms 138.86 ms]

Benchmarking Put/append: Warming up for 3.0000 s
Put/append              time:   [80.353 ms 80.538 ms 80.722 ms]

Actual integration into key paths should be done in separate PRs with proper testing and benchmarks. Some places may be able to simply fall back to put, while others may want to report an error for different handling if the key is unexpectedly not the highest.

@hai-rise hai-rise requested a review from rakita as a code owner September 21, 2025 14:25
@hai-rise hai-rise requested a review from joshieDo as a code owner September 21, 2025 14:25
@github-project-automation github-project-automation bot moved this to Backlog in Reth Tracker Sep 21, 2025
hai-rise
hai-rise commented Sep 21, 2025
View reviewed changes
mattsse
mattsse requested changes Sep 22, 2025
View reviewed changes
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some stlye nits, defer to @shekhirin on mdbx

crates/storage/db/src/implementation/mdbx/tx.rs
}

impl Tx<RW> {
fn put<T: Table>(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add some docs here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done 🙏.

crates/storage/db/src/implementation/mdbx/tx.rs
let key = key.encode();
let value = value.compress();
self.execute_with_operation_metric::<T, _>(
Operation::Put,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is putupsert different from just put?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's the same, just renamed for clarity. By default with flag 0, put will do an upsert

reth/crates/storage/libmdbx-rs/mdbx-sys/libmdbx/mdbx.h

Lines 1624 to 1625 in cff5512

/** Upsertion by default (without any other flags) */
MDBX_UPSERT = 0,

crates/storage/errors/src/db.rs
Comment on lines +109 to +112
/// Put upsert.
PutUpsert,
/// Put append.
PutAppend,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we add another enum for PutKind then this can replace the bool and we can do kind.into() instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Too many Intos are annoying, so I did a simple TxKind::into_operation_and_flags that covers everything.

@github-project-automation github-project-automation bot moved this from Backlog to In Progress in Reth Tracker Sep 22, 2025
shekhirin
shekhirin approved these changes Sep 22, 2025
View reviewed changes
Copy link
Collaborator

@shekhirin shekhirin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of this makes sense, nice catch that we can append not just for cursor, but for simple put operations too.

Agreed with Matt's nits

@shekhirin shekhirin added C-perf A change motivated by improving speed, memory usage or disk footprint A-db Related to the database labels Sep 22, 2025
@mattsse mattsse added this pull request to the merge queue Sep 24, 2025
Merged via the queue into paradigmxyz:main with commit 1a68d8e Sep 24, 2025
41 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Reth Tracker Sep 24, 2025
@hai-rise hai-rise deleted the mdbx-put-append branch September 24, 2025 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shekhirin shekhirin shekhirin approved these changes

@mattsse mattsse mattsse approved these changes

@rakita rakita Awaiting requested review from rakita rakita is a code owner

@joshieDo joshieDo Awaiting requested review from joshieDo joshieDo is a code owner

Assignees
No one assigned
Labels
A-db Related to the database C-perf A change motivated by improving speed, memory usage or disk footprint
Projects
Reth Tracker
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@hai-rise @shekhirin @mattsse